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After Action Reviews for Summer Learning Programs: 
Identifying Lessons Learned to Accelerate Learning 


Districts and schools are receiving once in a life-time funding through the American Recovery Plan 
(ARP) to address the learning needs of students over the next few years, especially those who have 
been most impacted by a year of remote or hybrid learning. State education agencies (SEAs) 
including the District of Columbia, Puerto Rico, outlying areas, freely associated states, and Tribal 
sovereign nations, and local education agencies (LEAs) across the country should be strategic in 
their use of funds to make a meaningful, real impact on accelerating student learning. This is 
especially important for those experiencing larger gaps as compared to their peers due to disrupted 
learning during the pandemic. 


After-Action Reviews (AARs) on summer learning programs, led by SEAs, LEAs, schools, or 
partnering organizations who provide summer learning services, can help users identify and apply 
lessons learned from this year’s summer recovery programs to accelerate learning in the future. 
Partners or vendors can hold their own AARs to improve their practice for future years and 
contracts as well. The AAR can be extremely helpful in identifying lessons learned from summer 
recovery and acceleration programs, which then inform strategic decisions for future school year 
and summer learning opportunities. 


What follows is an adaptation of an After-Action Review Guide and an After Action Review 
Guide for Learning Recovery that focuses specifically on key questions and data that SEAs and 
LEAs, including Tribes, can use to reflect on this summer’s learning recovery and acceleration 
programs. The findings can then be used by leaders to plan for school-year programs and 
subsequent summers. Applying lessons learned also helps guide the use of American Recovery 
Funds, so they are used strategically to achieve intended results. 


What is an AAR? 


An AAR is a method for extracting lessons from one action, event, or project, and applying the 
lessons learned to others. It enables a team to analyze for themselves what happened, why it 
happened, and how to use learnings to sustain strengths and improve weaknesses. The AAR cycle is 
a four-step process, as displayed in figure 1. 
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Figure 1. The AAR cycle 
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Critical elements of an effective AAR include 


» engaging in candid discussions without blame or judgment; 
» including specific examples with evidence; and 
» identifying specific actions to apply lessons learned in a different context or situation. 
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Step 1. Preparation 


Ideally, an AAR might include stakeholders that represent the SEA, LEA, school, and/or external vendor(s) to complete a more 


comprehensive review of the programs and to identify areas for improvement. 


Preparing for an AAR is just as important as conducting the review. Ideally, the review should be conducted in person whenever local 
health and safety conditions allow, however, it could be accomplished virtually as well. Keep in mind, the context of SEAs, LEAs, and schools 


differ and so should the some of the components of preparation. Table 1 provides suggestions for an SEA, LEA, and school to effectively 


plan for a facilitated AAR. 


Table 1. After-Action Review preparation 


Components 


SEA 


LEA 


School or Service Provider 


Logistics* 


» Select a room or location that is conducive to face-to-face discussions, or if virtual, breakout rooms for small group 


discussions followed by whole group sharing 


» Arrange for technology to capture discussions and decisions in real time so that all can view 


» Identify a date and time that allows for all designated participants to attend 


» Prepare an agenda that allows adequate time for full and inclusive discussion among all participants and incorporates 


breaks 


» Share the agenda and any relevant data or information prior to the scheduled meeting 
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representatives 


organizations, other agencies, 
or businesses 


Components SEA LEA School or Service Provider 
Participants” » SEA leaders and staff » LEA leaders and staff including: | >» School and provider leaders and staff including: 
including: >» Superintendent >» School administrators 
>» Chief State School Officer >» Director of Finance, including >» Teacher representatives (e.g., grade level, 
» Chief Operating Officer procurement staff subject, or department) 
>» Chief Academic Officer > Director of Special Education >» Support staff 
>» Student Services > Director of Student Services > Facilities staff 
> Staff overseeing federal > Director of Transportation >» Nutrition staff 
grants > Director of Food Services > Transportation staff 
>» Program or project leaders > Director of Facilities » Community Representatives 
» Partners (e.g., other agencies, | »» Community Representatives » Student representatives, as age appropriate 
businesses) who assisted in » School representatives » Provider or vendor representatives and staff, if 
service delivery » Key administrative and program applicable 
» Superintendent and school staff from partner » Key administrative and program staff from 


partner organizations, other agencies, or 
businesses 


Roles (see Role 
Descriptions 
below) 


» Facilitator 
» Notetaker 
» Timekeeper 


Data 


» Policy changes made and impact of changes 


» Cost effectiveness of programs and services provided by the school or district, as well as vendors or partners? 


» Staffing or recruitment data related to summer programming 


» Professional development programs implemented to support the program 


» Indicators of outcomes tied to program goals 
» Enrollment data related to summer programming 


» Infrastructure improvements made with funds, maintenance costs associated with the improvements, current 
assessment of infrastructure improvements as well as use (e.g., funds used to expand internet access, past and current 


usage data) 


impact. 


Logistics include time, location, materials, and methods. 


and example of cost effectiveness analysis. 
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Role Descriptions 


A designated facilitator allows all participants involved in the event to be present and engaged throughout the process. The facilitator 
keeps the conversations and the process flowing. The facilitator strikes a balance of preventing unproductive digressions, while allowing 
for productive conversation that is not explicitly planned. A notetaker captures notes on the discussions and documents key decisions. If 
the meeting is more than 1-hour, multiple notetakers are recommended. A timekeeper Keeps the process flowing based on approximate 
time allotted for each question and works with the facilitator to make sure all agenda items are covered. 


Ground Rules 


Ground rules should be established to ensure discussions are honest, respectful, and productive. Organizations can use their existing 
norms, or a set that the team agrees upon at the beginning of the review. Possible ground rules include 


» Participate actively 

» Value all points of view 

» Use “Yes...and,” rather than either/or thinking 

» Use consensus building to arrive at decisions 

» Focus on issues that can be addressed, not on issues outside the organization’s sphere of influence 
» Focus on improving performance, not placing blame 

» Acknowledge one’s own errors or mistakes 


Step 2. Conducting the AAR 


The AAR questions! in table 2 focus on critical areas of planning to provide equitable, effective programs and services. The services should 
focus on accelerating student learning and assist in recovery from the impact of disrupted learning caused by COVID-19. The questions 
are designed to guide discussion to identify lessons learned from this summer’s learning recovery efforts so that the information can then 
be used in Step 3 to address immediate recommendations and applied to other projects to improve performance and results. Actions that 
worked well can be applied to current planning for the school year and subsequent summer learning recovery and acceleration programs. 
For example, if following an established flow of communication was identified as having a positive effect on team performance, then the 


1 Questions were adapted from the After-Action Review Guide (Layland et al., 2020) 
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same communication flow can be applied to planning subsequent summer learning recovery and acceleration planning. However, if there 
were missteps in the procurement process for external providers, then the missteps are identified and avoided in future planning. 


Table 2. Summer learning AAR questions 


Focus Areas 


SEA LEA School Service Provider 


Overall/General 


» What was the expected outcome for use of state and federal funds for summer learning and acceleration? 
> What were the purpose, objectives, targeted audience, time frame, and intended outputs and outcomes? 
> What were the anticipated barriers or challenges and how did you plan for them? 
» What actually happened as a result of the funding and summer learning programming? 
» What went well and what did not work (i.e., What caused our results?)? 
> What were our successes and what made them successful? 
>» What didn’t work well and what caused those missteps? 
>» What did or can we learn from these successes and missteps? 
» What will we do to adapt or refine our executions for a better outcome for future school year and summer learning 
programs? 
> Given the information and knowledge we had at the time, what did we do or should have done to sustain the success 
experienced? 
>» Given the information we have now, what do we need to do to sustain our successes moving forward? 
>» Given the information and knowledge we had at the time, what could have been done better? 
> Given the information we have now, what can we do differently moving forward? 
> What advice could be given to a leader based on the experiences and discussions we have had? 
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Focus Areas SEA LEA School Service Provider 
Policy » What policy changes » What district policies or » What school procedural » What provider procedural 
were made as a result of procedures changes were changes were made asa changes were made to 
the focus of and the made as a result of the focus result of the focus of and partner and provider 
need for summer of and the need for summer the need for summer services to address the 
learning and learning and acceleration? learning and acceleration? need for summer learning 
acceleration? » Are those changes still in » Are those changes still in and acceleration? 
» Are those changes still in place? place? » How could request for 
place? » What impact did or do the » What impact did or do the proposal (RFP) and contract 
» What impact did or do policies have on student school procedures have on procedures be improved to 
the policies have on learning? Did the results match student learning? Did the reduce the number of 
student learning? Didthe] expectations? Why or why results match expectations?} changes needed and to 
results match not? Why or why not? better support the 
expectations? Why or » Were there additional policy » Were there additional partnership and provision 
why not? barriers that we should have practice barriers that we of programs or services? 
» Were there additional addressed? should have addressed? 
policy barriers that we » How could policies be revised | >» How could procedures or 
should have addressed? or added to better support practice be revised or 
» How could policies be summer learning and added to better support 
revised or added to acceleration? summer learning and 
better support summer acceleration? 
learning and 


acceleration? 
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Focus Areas 


SEA 


LEA 


School 


Service Provider 


Finance 


» What role did the SEA 
take in providing 
guidance to LEAs about 
use of state and federal 
funding for summer 
learning programs? 

» What supports did the 
SEA provide to LEAs 
about the procurement 
of external vendors? 

» Did the SEA use lessons 
learned from prior large 
federal funding pools to 
plan guidance for the 
current efforts? 

» Did local efforts meet 
our expected outcomes? 
Why or why not? 

» What lessons were 
learned about using 
federal relief funding to 
provide targeted student 
supports? 

» How could the lessons 
be used to improve 
finance policies and 
procedures to better 
support summer 
learning and 
acceleration? 


» 


» 


» 


» 


How were state and federal 
funds utilized to support 
summer learning 
programming? 

How have budget 
decisions/priorities targeted 
equitable summer learning 
services for the most impacted 
students? 

Were summer learning 
programs funded sufficiently 
to address student needs? 
What lessons were learned 
about using federal funding to 
provide targeted student 
supports? 

How can we use lessons 
learned from this summer to 
plan for next summer’s and 
the next school year’s learning 
recovery programs? 


» Does the school manage its 
own budget for summer 
learning programming? 

» How was funding used to 
lessen the short-term 
impact on instruction? 
Long-term impact? What 
actions were successful in 
mitigating the impact? 
What actions were 
ineffective? How did you 
know? 

» What lessons were learned 
about using federal funding 
to provide targeted student 
supports? 

» How can we use lessons 
learned from this summer 
to plan for next summer’s 
and the next school year’s 
learning recovery 
programs? 


» Were there any contractual 
challenges or changes due 
to the focus on providing 
summer learning? 

» How could the contract 
process be improved to 
better support program or 
service delivery and 
effectiveness? 
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Focus Areas 


SEA 


LEA 


School 


Service Provider 


Infrastructure/ 
Capital 


» What infrastructure 
improvements or 
changes were made to 
support summer 
learning programming 
and why? 

» What was the 
anticipated impact of the 
improvements? What 
was the actual 
immediate and long- 
term impact of the 
improvements? Have 
any additional costs 
occurred because of the 
improvements (e.g., 
upgrades, 
maintenance)? 

» Are additional 
infrastructure 
improvements needed 
to address barriers to 
implementation of 
effective summer 
learning programs? 

» How can we use the 
lessons to address 
current infrastructure 
concerns in a sustainable 
way? 


» 


» 


» 


What infrastructure 
improvements or changes 


were made to support summer 


learning programming and 
why? 

What was the anticipated 
impact of the improvements? 
What was the actual 
immediate and long-term 
impact of the improvements? 
Have any additional costs 
occurred because of the 


improvements (e.g., upgrades, 


maintenance)? 

Are additional infrastructure 
improvements needed to 
address barriers to 
implementation of effective 
summer learning programs? 


How can we use the lessons to 
address current infrastructure 
concerns in a sustainable way? 


» 


» 


» 


» 


What infrastructure 
improvements or changes 
were made to support 
summer learning 
programming and why? 
What was the anticipated 
impact of the 
improvements? What was 
the actual immediate and 
long-term impact of the 
improvements? Have any 
additional costs occurred 
because of the 
improvements (e.g., 
upgrades, maintenance)? 
Are additional 
infrastructure 
improvements needed to 
address barriers to 
implementation of effective 
summer learning 
programs? 

How can we use the lessons 
to address current 
infrastructure concerns ina 
sustainable way? 


» 


» 


» 


What infrastructure 
improvements or changes 
were needed to support 
summer learning 
programming and why? 
What infrastructure 
changes were needed to 
better support the program 
or service delivery? 

What could be changed in 
the vendor process and 
partnership that would 
avoid any infrastructure 
issues in future program 
partnerships? 
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Focus Areas 


SEA 


LEA 


School 


Service Provider 


Partnerships 


» What partnerships have been used in the past to provide summer learning programs? What partnerships were used this 


year? 


» Are there community organizations already providing accelerated learning (e.g., high dosage tutoring or summer camps)? 
How effective are those programs? How could partnerships be formed to leverage these services and existing providers? 


» 


— 


summer learning? 
Y 
> 
» 


ee Ne 


How efficient were the contracting/bid processes? How could these be improved to facilitate strong partnerships for 


How did the members of a partnership communicate with each other to coordinate services for students? 
How were partnerships routinely monitored to make mid-course corrections, as data and implementation indicated? 
How were partnerships evaluated for effectiveness, including breakdowns by student subgroups? 


» What components need to be in place to monitor and evaluate the effectiveness of each partnership as well as any services 


provided by the partners? 


Staffing and 
Roles 


» What role did the SEA 
play in allocating and 
using funds for summer 
learning programming? 

» What role did the SEA 

play in improving the 

quality of summer 
learning services offered 

(e.g., developing 

preferred provider lists 

creating selection criteria 
or sample RFPs, etc.)? 

Did the SEA have the 

staffing capacity to 

support LEAs develop 
and implement summer 
programming? 

How did the SEA 

reorganize staffing to 

support this work? 

How were funds used to 

address staffing capacity 


») 


- 


y 


~ 


y 


~ 


» What role did the LEA play in 


allocating and using funds for 
summer learning 
programming? 

What role did the LEA play in 
improving the quality of 
summer learning services 
offered (e.g., running an RFP 
process, monitoring and 
evaluating vendors, etc.)? 
Did the LEA have the staffing 
capacity to support schools 
and vendors develop and 
implement summer 
programming? 

How did the LEA reorganize 
staffing to support this work? 
How were funds used to 
address staffing capacity for 
summer learning? 


» How can we use the lessons 


from staffing for summer 


~ 


~ 


al 


~ 


~ 


~ 


What was school staffing 
before the pandemic? 

How was staffing impacted 
by the pandemic? 

How were funds used to 
address staffing capacity for 
summer learning? 

What was learned about 
using relief funds to 
address staffing capacity to 
provide summer learning? 
How can we use the lessons 
to address current staffing 
capacity and improve 
diverse and equitable 
staffing plans for continued 
expanded learning 
opportunities in the future? 
How were families or 
caregivers engaged in the 
program or service? 


» How was the partnership 
structured in a way that 
would lead to success? 
How was the program 
staffed to deliver needed 
services? 

Were there any staffing 
barriers that impacted 
service delivery? If so, how 
could they be addressed in 
the future? 

What structural barriers or 
challenges were 
experienced that impacted 
delivery and effectiveness 
of provided services? 

How could the partnership 
be better structured to be 
more effective in providing 
services and get better 
results? 


y 


~ 


» 


~ 


y 


Lg 


» 


~ 
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Focus Areas 


SEA 


LEA 


School 


Service Provider 


to assist districts and 
schools in providing 
summer learning? 


learning to address current 
staffing capacity and improve 
diverse and equitable staffing 


» 


How did the service 
providers communicate 
with families or caregivers? 


» How were families or 
caregivers engaged in the 
program or service? 


or programs were added 
to support summer 

learning? How impactful 
were those programs on 


support summer learning? 
How impactful were those 
programs on instruction and 
student learning? 


» 


delivery? 

What additional services or 
programs were added to 
support summer learning? 
How impactful were those 


» What was learned about as the district plans for How effective was the » How did the service 
using relief funds to continued expanded learning communication? providers communicate 
address staffing capacity opportunities in the future? » How could the role of with families or caregivers? 
to support districts and parents or caregivers be How effective was the 
schools in providing strengthened to better communication? 
summer learning? support attendance, » How could the role of 

» How can we use the participation, and learning? parents or caregivers be 
lessons to address strengthened to better 
current staffing capacity support attendance, 
and improve diverse and participation, and learning? 
equitable staffing for the » How could communication 
future? between the providers, 

parents, families, and 
schools and districts be 
strengthened? 
Services or » How were decisions » How were decisions made » How were decisions made _ | »» What were the academic or 
Programs made related to the related to the delivery of related to the delivery of curricular components of 
delivery of summer summer learning programs summer learning programs the program or services? 
learning programs and and services? and services? How did these align to 
services? »» What data were used to » What data were used to student needs? 

» What data were used to inform decisions? What inform decisions? What » How effective were the 
inform decisions? What considerations were given to considerations were given academic or curricular 
considerations were equity? to equity? components in addressing 
given to equity? » What additional services or » Was the program or service learning or acceleration 

» What additional services programs were added to equitable in access and needs? 


» What questions or 
information did you not 
ask/not have that would 
have enabled you to 
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» 


» 


» 


» 


» 


What data do you wish 
you had had to make 
programmatic decisions? 
How did you correct that 
data void? 

What data are available 
to make decisions 
related to summer 
learning? 

How are programmatic 
priorities established? 
How are programs and 
services selected to 
address priorities? 

What guidance was 
provided to districts and 
schools on use of funds 
for summer learning 
programs and services 
and the evaluation of 
those services and 
programs? 

What lessons learned 
could assist in selecting, 
implementing, and 
evaluating programs and 
services to recover and 
accelerate learning for 
students most in need? 


» 


programmatic decisions? How 
did you correct that data void? 
What data are available now 
to make decisions related to 
summer learning? 

How are programmatic 
priorities established? How are 
programs and services 
selected to address priorities? 
What guidance was provided 
to districts on the use of funds 
for summer learning programs 
and services and the 
evaluation of those services 
and programs? Was the 
guidance useful? What 
additional guidance or support 
would have been helpful? 
What lessons learned could 
assist in selecting, 
implementing, and evaluating 
programs and services to 
accelerate learning for 
students most in need? 


» 


» 


» 


» 


» 


Were any components of 
the program designed to 
address attendance, 
behavioral, or social 
emotional learning needs? 
If yes, how effective were 
the components? 

What data do you wish you 
had had to make 
programmatic decisions? 
How did you correct that 
data void? 

What data are available 
now to make decisions 
related to summer 
learning? 

How are programmatic 
priorities established? How 
are programs and services 
selected to address 
priorities? 

What guidance was 
provided to schools on the 
use of funds for summer 
learning programs and 
services and the evaluation 
of those services and 
programs? Was the 
guidance useful? What 
additional guidance or 
support would have been 
helpful? 


» 


» 


» 


Focus Areas SEA LEA School Service Provider 
instruction and student | >» What data do you wish you programs on instruction develop an even stronger 
learning? had had to make and student learning? program? 


Was the program or service 
equitable in access and 
delivery? 

Were any components of 
the program designed to 
address attendance, 
behavioral, or social 
emotional learning needs? 
If yes, how effective were 
the components? 

How could the program or 
service be improved for 
next summer or school 
year? 


www.compcenternetwork.org 


After Action Reviews for Summer Learning Programs: Identifying Lessons Learned to Accelerate Learning 


Focus Areas 


SEA 


LEA 


School 


Service Provider 


» What lessons learned could 
assist in selecting, 
implementing, and 
evaluating programs and 
services to accelerate 
learning for students most 
in need? 


Vendors or 
Providers 


» 


» 


» 


» 


» 


Did the SEA use or 
partner with any 
providers or vendors to 
support districts and 
schools in providing 
summer learning? How 
were they used? 

What guidance was 
provided to districts and 
schools related to using 
vendors or providers to 
deliver summer learning 
or additional learning 
opportunities? 

How was the guidance 
evaluated? 

How were the 
partnership or services 
provided by the vendor 
or provider evaluated? 
What lessons learned 
could be applied to 
guidance and planning 
for continued expanded 
learning opportunities? 


» 


» 


» 


» 


» 


What guidance was provided 
to schools related to using 
vendors or providers for 
summer learning? How was 
the guidance evaluated? 
What guidance was received 
by the state related to using 
vendors or providers for 
summer learning? How was 
the guidance used to plan and 
provide summer learning? 
Were vendors or providers 
used to deliver summer 
learning opportunities? How 
were they used? 

How were the partnership or 
services provided by the 
vendor or provider evaluated? 
What lessons learned could be 
applied to guidance and 
planning for continued 
expanded learning 
opportunities? 


» What guidance did the 
district provide to the 
schools related to using 
vendors or providers for 
summer learning? How was 
the guidance used to plan 
and provide summer 
learning? 

» Were vendors or providers 
used to deliver summer 
learning opportunities? 
How were they used? 

» How were the partnership 
or services provided by the 
vendor or provider 
evaluated? 

» What lessons learned could 
be applied to guidance and 
planning for continued 
expanded learning 
opportunities? 


» How did the LEA coordinate 
and communicate with any 
and all vendors/providers 
on an ongoing basis? 

» Did the LEA meet any 
vendor-specified non- 
negotiables outlined in the 
contract or amemorandum 
of understanding (MOU)? 
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Focus Areas 


SEA 


LEA 


School 


Service Provider 


Monitoring and 
Evaluation 


» How were the use of 
funds for summer 
learning and 
effectiveness tracked 
and evaluated? 

» What data would you 
like to have had to make 
decisions on use of 
stimulus funds? 

» What data would assist 
in making planning 
decisions now? What 
actions can be taken 
now to ensure the right 
data are available to 
inform future decisions 
to address impact of 
disruptions in learning? 

» Have you remedied any 
data voids? Why or why 
not? 

» What lessons learned 
could assist in improving 
evaluation of actions 
taken to provide 
programs and services to 
accelerate learning for 
students most in need? 


» 


» 


» 


How were the use of funds for 
summer learning and 
effectiveness tracked and 
evaluated? 

How were the effectiveness of 
summer learning programs 
tracked and evaluated? 

How was equity of service 
delivery and outcomes 
evaluated? 

What data would you like to 
have had to make decisions on 
use of stimulus funds? 

Have you remedied any data 
voids? Why or why not? 

What data would assist in 
making planning decisions 
now? What actions can be 
taken now to ensure the right 
data are available to inform 
future decisions to address 
impact of disruptions in 
learning? 

What lessons learned could 
assist in improving evaluation 
of actions taken to provide 
programs and services to 
accelerate learning for 
students most in need? 


» How were the use of funds 
for summer learning and 
effectiveness tracked and 
evaluated? 

» How was the impact of the 
summer learning programs 
on individual students 
assessed and analyzed? 

» How was equity of service 
delivery and outcomes 
evaluated? 

» What data would you like 
to have had to make 
decisions on use of stimulus 
funds? 

» Have you remedied any 
data voids? Why or why 
not? 

» What data would assist in 
making planning decisions 
now? What actions can be 
taken now to ensure the 
right data are available to 
inform future decisions to 
address impact of 
disruptions in learning? 

» What lessons learned could 
assist in improving 
evaluation of actions taken 
to provide programs and 
services to accelerate 
learning for students most 
in need? 


» What was used to measure 
implementation 
effectiveness? Did the 
measure provide timely 
results so adjustments 
could be made? Were the 
adjustments effective? 

» How was implementation 
measured? How effective 
was the measure in 
identifying implementation 
issues in a timely manner? 

» How was student impact 
measured? Was evidence 
of impact examined across 
demographic and 
subgroups of students? 

» How aligned were the 
measures to the 
expectations set by the 
state, district, or school? 

» How were results shared 
with the state, district, or 
school? Was the format 
accessible and user 
friendly? 

» How could implementation 
and impact measures be 
improved? 

» How could communication 
of implementation and 
student results be more 
effective? 
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After Action Reviews for Summer Learning Programs: Identifying Lessons Learned to Accelerate Learning 


Step 3. Using AAR Results for Planning to Accelerate Learning 


Documenting the lessons learned is just that—documentation. Learning has not really occurred unless there is a change in behavior. We 
must apply the lessons learned and continually adapt to improve actions and sustain the practices that work. AAR teams can use the 
information gathered through the AAR to build possible scenarios for learning recovery and acceleration. More information on Scenario 


Planning for Learning Recovery is available here or information on Scenario Planning in general is available here. 


Table 3. Lessons learned 


Focus Areas 


Successes 


Missteps 


Implications for Current Planning 


Overall/General 


Policy 


Finance 


Infrastructure/ 
Capital 


Partnerships 


Staffing 


Services or 
Programs 


Monitoring and 
Evaluation 
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